Revisiting Consistency Regularization for Semi-Supervised Learning

Abstract Consistency regularization is one of the most widely-used techniques for semi-supervised learning (SSL). Generally, aim to train a model that invariant various data augmentations. In this paper, we revisit idea and find enforcing invariance by decreasing distances between features from differently augmented images leads improved performance. However, encouraging equivariance instead, increasing feature distance, further improves To end, propose an consistency framework simple yet effective technique, FeatDistLoss, imposes on classifier level, respectively. Experimental results show our defines new state art across variety standard benchmarks as well imbalanced benchmarks. Particularly, outperform previous work significant margin in low regimes at large imbalance ratios. Extensive experiments are conducted analyze method, code will be published.